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LETTER OF PROMULGATION 


STATEMENT 


The — enclosed NATO _ standardization 
agreement (STANAG), which has been ratified 
by member nations, as reflected in the 
NATO Standardization Document Database 
(NSDD), is promulgated herewith. 


ENACTMENT 


This STANAG is effective upon receipt for use 
by the participating nations and NATO bodies. 


ACTIONS BY NATIONS 


Nations are invited to examine their ratification 
of the STANAG and, if they have not already 
done so, advise the NSO of their intention 
regarding its ratification and implementation. 


Once implemented, Allies shall provide 
implementation details through the electronic 
reporting tool. 


SECURITY CLASSIFICATION 


This STANAG is a NATO non-classified 
document to be handled in accordance 
with C-M(2002)60. 


RESTRICTION TO REPRODUCTION 


No part of this publication may be reproduced, 
stored in a_ retrieval system, used 
commercially, adapted, or transmitted in any 
form or by any means, electronic, mechanical, 
photocopying, recording or otherwise, without 
the prior permission of the publisher. With the 
exception of commercial sales, this does not 
apply to member and partner nations, 
or NATO commands and bodies. 


NSO/0043(2019)WG2/4107 


LETTRE DE PROMULGATION 


DECLARATION 


L'accord de normalisation OTAN (STANAG) 
ci-joint, qui a été ratifié par les pays membres 
dans les conditions figurant dans la Base de 
données des documents de 
normalisation OTAN (NSDD), est promulgué 
par la présente. 


ENTREE EN VIGUEUR 
Ce STANAG entre en vigueur dés réception aux 


fins d’application par les pays et les 
organismes OTAN participants. 

MESURES A PRENDRE PAR LES PAYS 

Les pays sont invités a examiner l'état 


d'avancement de la ratification du STANAG eta 
informer, s’ils ne l’ont pas encore fait, le NSO de 
leur intention concernant sa ratification et sa 
mise en application. 


Dés que le STANAG est mis en application, les 
Alliés doivent fournir les informations y 
afférentes via l’outil de notification électronique. 


CLASSIFICATION DE SECURITE 


Ce STANAG est un document OTAN 
non classifié qui doit étre traité conformément 
au C-M(2002)60. 


RESTRICTION DE REPRODUCTION 


Aucune partie de cette publication ne peut étre 
reproduite, incorporée dans une base 
documentaire, utilisée commercialement, 
adaptée ou transmise sous quelque forme ou 
par quelque moyen que ce soit (électronique, 
mécanique, photocopie, enregistrement ou 
autre), sans l'autorisation préalable de I’éditeur. 
Sauf pour les ventes commerciales, cela ne 
s’applique pas aux Etats membres ou aux pays 
partenaires, ni aux commandements et 
organismes de l'OTAN. 


ADDITIONAL INFORMATION 


This Edition of STANAG 4107 reflects: 
4. the cancellation of AQAP-2009, -2120 
and -2130; and 


2. the ratification of AQAP-2105, Edition C, 
which has been reviewed and updated. 


Nations are asked to note that AQAP-4107 has 
been modified to reflect these changes but the 
nature of the agreement for mutual 
Government Assurance and use of AQAPs has 
not changed. 


This edition of STANAG 4107 addresses the 
cancellation of contractual AQAPs that are 
based on 1S0 9001:2008 which has been 
superseded and was withdrawn by ISO in 
September 2018. In preparation for this, 
AC/327 Working Group 2 (WG/2) developed 


contractual AQAPs that align 
with ISO 9001:2015 and issued 
implementation guidance for acquirers, 


suppliers and Government Quality Assurance 
Representatives (GQARs). Implementation 


details were published in September 2016 as 
and 


AQAP-2110-SRD.1 (Transition 
implementation Guidance). 


Zoltan GULYAS 
Brigadier General, HUNA\ 
Director, NATO Standardization Office 


INFORMATIONS SUPPLEMENTAIRES 
Cette édition du STANAG 4107 tient compte : 


1. de l'annulation des AQAP n° 2009, 2120 
et 2130; 


2. et de la ratification de lAQAP-2105, 
Edition C, qui a été revue et mise a jour. 


Les pays sont priés de noter que l' AQAP-4107 
a été madifiée pour rendre compte de ces 
changements, mais la nature de l'accord pour 
les services mutuels d’assurance officielle de la 
qualité et l'utilisation des AQAP n'a pas changé. 


Cette édition du STANAG 4107 couvre 
Vannulation des AQAP contractuelles basées 
sur la norme ISO 9001:2008, laquelle a été 
annulée et remplacée, puis retirée par I'lSO en 
septembre 2018. En vue de ce retrait, le Groupe 
de travail 2 (WG/2) de 'AC/327 a mis au point 
des AQAP contractuelles alignées sur la 
norme iSO 9001:2015 et a publié des directives 
de mise en ceuvre pour les acquéreurs, les 
fournisseurs et les représentants pour 
l'assurance officielle de fa qualité (RAOQ) sous 
la cote AQAP-2110-SRD.1 (Lignes directrices 
pour fa transition et fa mise en ceuvre), en 
septembre 2016, 


4, ~~ 
aie 
Zoltan GULYAS 
Général de brigade aérienne, HUNAF 
Directeur du Bureau OTAN de normalisation 
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MUTUAL ACCEPTANCE OF 
GOVERNMENT QUALITY ASSURANCE 
AND USAGE OF THE ALLIED QUALITY 
ASSURANCE PUBLICATIONS (AQAP) 


AIM 


The aim of this NATO standardization 
agreement (STANAG) is to respond to the 
following interoperability requirements. 


INTEROPERABILITY REQUIREMENTS 


To set forth the process, procedures, terms and 
conditions under which Mutual Government 
Quality Assurance of defence products is to be 
performed by the appropriate national authority 
of one NATO member nation, at the request of 
another NATO member nation or 
NATO organization; and to standardize the 
development, updating and application of AQAP 
on the basis of the concept of quality assurance 
in the procurement of defence products. 


AGREEMENT 


Participating nations agree to implement the 
following standards. 


STANDARDS 


AQAP-2000, 
AQAP-2070, 
AQAP-2105, 
AQAP-2110, 
AQAP-2131, 
AQAP-2210, 
AQAP-2310, Edition B 
AQAP-4107, Edition A 


OTHER RELATED DOCUMENTS 
None. 
SUPERSEDED DOCUMENTS 


This STANAG 
document: 


STANAG 4107, Edition 10, dated 
18 December 2017 


Edition 3 
Edition B 
Edition C 
Edition D 
Edition C 
Edition A 


supersedes the following 


ACCEPTATION DE SERVICES 
MUTUELS D’ASSURANCE OFFICIELLE 
DE LA QUALITE (AOQ) ET UTILISATION 
DES PUBLICATIONS INTERALLIEES 
SUR L’ASSURANCE DE LA QUALITE 
(AQAP) 


BUT 


Le présent accord de normalisation OTAN 
(STANAG) a pour but de répondre aux 
exigences d’interopérabilité suivantes. 


EXIGENCES D’INTEROPERABILITE 


Définir les processus, procédures, modalités et 
conditions régissant l’exercice mutuel de 
lassurance officielle de la qualité des produits 
de défense par les autorités nationales 
compétentes d’un pays de l'OTAN, a la requéte 
d'un autre pays de lOTAN ou d'un 
organisme OTAN ; et normaliser I’élaboration, la 
mise a jour et la mise en application des AQAP, 
a partir du concept d'assurance de la qualité 
applicable a I'acquisition des produits de 
défense. 


ACCORD 


Les pays participants conviennent de mettre en 
application les normes suivantes. 


NORMES 


AQAP-2000, 
AQAP-2070, 
AQAP-2105, 
AQAP-2110, 
AQAP-2131, 
AQAP-2210, E 
AQAP-2310, Edition B 
AQAP-4107, Edition A 


AUTRES DOCUMENTS CONNEXES 
Aucun. 
DOCUMENTS ANNULES ET REMPLACES 


Le présent STANAG annule et remplace le 
document suivant : 


STANAG 4107, Edition 10, du 
18 décembre 2017 


Edi 
Edi 
Edi 
Edi 
Edi 
Edi 


ion 3 
ion B 
ion C 
ion D 
ion C 
ion A 


NATIONAL RATIFICATION RESPONSE 


National responses are recorded in 
the NATO Standardization Document 
Database (NSDD). 


Allies shall provide ratification details through 
the electronic reporting tool (e-Reporting). 


IMPLEMENTATION OF THE AGREEMENT 


Where contractual quality assurance (QA) 
requirements are appropriate, NATO nations 
and bodies are required to use the updated 
AQAPs-2110, -2131 or -2310. 

This Edition of STANAG 4107 covers the 
release of AQAP-2105, Edition C. Nations are 
requested to use this contractual QA 
requirement when supplier quality plans are 
considered necessary. 


Partner nations are invited to use the QA 
publications covered by this agreement and to 
note that the provision of mutual GQA under 
this STANAG is reserved for NATO nations and 
agencies. 


Allies and NATO bodies shall provide 
implementation details through the electronic 
reporting tool (e-Reporting). 


Partner nations are invited to provide their 
implementation details through the electronic 
reporting tool (e-Reporting). 

NATO EFFECTIVE DATE (NED) 

Not applicable. 

REVIEW 


This STANAG is to be reviewed at least once 
every five years. The result of the review is to be 
recorded within the NSDD. 

TASKING AUTHORITY 


This STANAG is supervised under the authority 
of: 


REPONSES NATIONALES AUX DEMANDES 
DE RATIFICATION 


Les réponses nationales sont consignées dans 
la Base de données des documents de 
normalisation OTAN (NSDD). 

Les Alliés doivent rendre compte de leurs 
ratifications via de |'outil de notification 
électronique (e-Reporting). 


MISE EN APPLICATION DE L’ACCORD 


Lorsque des  exigences _ contractuelles 
d'assurance qualité sont appropriées, les pays 
et organismes de |'OTAN sont tenus d’appliquer 
les dispositions des éditions mises a jour 
des AQAP-2110, -2131 ou -2310. 

La présente édition du STANAG 4107 couvre la 
diffusion de I'AQAP-2105, Edition C. Les pays 
sont priés d'utiliser cette exigence contractuelle 
d'assurance qualité lorsque des plans qualité 
des fournisseurs sont jugés nécessaires. 


Les pays partenaires sont invités a utiliser les 
publications sur lassurance de la qualité 
couvertes par le présent accord et a noter que 
la fourniture de services mutuels d'AOQ dans le 
cadre de ce STANAG est réservée aux pays et 
agences de I'OTAN. 


Les Alliés et les organismes OTAN doivent 
rendre compte de leur mise en application via 
Poutil de notification électronique (e-Reporting). 


Les pays partenaires sont invités a rendre 
compte de leur mise en application via l’outil de 
notification électronique (e-Reporting). 

DATE D’ENTREE EN VIGUEUR OTAN (NED) 
Sans objet. 

REEXAMEN 


Le présent STANAG doit étre réexaminé au 
moins une fois tous les cinq ans. Le résultat de 
ce réexamen doit étre consigné dans la NSDD. 


AUTORITE DE TUTELLE 


Le présent STANAG est sous la responsabilité 
de: 


CNAD LIFE CYCLE MANAGEMENT GROUP/ 
GROUPE DE LA CDNA SUR LA GESTION DU CYCLE DE VIE 
(AC/327) 


WORKING GROUP 2 ON QUALITY/ __ 
GROUPE DE TRAVAIL 2 SUR LA QUALITE 
(WG/2) 


FEEDBACK INFORMATIONS EN RETOUR 
Any comments concerning this STANAG shall Tous les commentaires concernant le 


be directed to: présent STANAG doivent étre adressés au : 
NATO Standardization Office Bureau OTAN de normalisation 
(NSO) (NSO) 


Boulevard Léopold IIll 
1110 BRUXELLES — Belgique 


NATO STANDARD 
AQAP-2210 


NATO SUPPLEMENTARY SOFTWARE 
QUALITY ASSURANCE REQUIREMENTS 
TO AQAP-2110 OR AQAP-2310 


Edition A Version 2 


September 2015 


NORTH ATLANTIC TREATY ORGANIZATION 
ALLIED QUALITY ASSURANCE PUBLICATION 


Published by the 
NATO STANDARDIZATION OFFICE (NSO) 
© NATO/OTAN 


NORTH ATLANTIC TREATY ORGANIZATION (NATO) 
NATO STANDARDIZATION OFFICE (NSO) 
NATO LETTER OF PROMULGATION 


4 September 2015 


i, The enclosed Allied Quality Assurance Publication AQAP-2210, Edition A, 
Version 2 “NATO Supplementary Software Quality Assurance Requirements to 
AQAP-2110 or AQAP-2310", which has been approved by the nations in the Life 
Cycle Management Group (AC/327), is promulgated herewith. The agreement of 
nations to use this publication is recorded in STANAG 4107. 


2: AQAP-2210, Edition A, Version 2 is effective upon receipt and supersedes 
AQAP-2210 Edition 1, which shall be destroyed in accordance with the local 
procedure for the destruction of documents. 


3. No part of this publication may be reproduced, stored in a retrieval system, 
used commercially, adapted, or transmitted in any form or by any means, electronic, 
mechanical, photo-copying, recording or otherwise, without the prior permission of 
the publisher. With the exception of commercial sales, this does not apply to member 
or partner nations, or NATO commands and bodies. 


4. This publication shall be handled in accordance with C-M(2002)60. 


FS Mowills 


Edvardas MAZEIKIS 
Major General, LTUAF 
Director, NATO Standardization Office 
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FOREWORD 


The Acquirer's quality assurance requirements stated in this document, are based on 
the experience that quality management of the entire software development process 
is the key to achieving software quality in complex and mission critical computer 
systems such as weapon systems, communication systems, and command and 
control systems. To ensure the quality of the software development process, such 
processes must be planned, controlled and improved, with the aim of reducing, 
eliminating and, most importantly, preventing software quality deficiencies. 


In accordance with international standardization, functional rather than organizational 
definitions for software quality management are used to avoid problems introduced 
by traditional quality concepts and their organizational boundaries. This publication, 
therefore, is not specifically addressed to software quality organizations, but rather to 
the overall organizational structure and the different management levels involved in a 
software project. 


This publication is designed for use in contracts, and defines the requirements for the 
Software Quality Management Activities as related to the Project to be documented 
in a Software Project Quality Plan. These activities are based on the Supplier's 
Software Quality System. The publication also requires the evaluation of the 
Software Quality Management Activities to ensure their effectiveness. 


The application of this publication is not restricted to any particular type or form of 
software. This publication does not specify any particular software development 
model, nor does it stipulate which software development methods should be used. 
This publication allows flexibility in adapting the required documentation and 
procedures to the specific development and procurement processes of the project. 


This publication supersedes AQAP 2210 Edition 1, and is intended for use with 
AQAP 2110 or AQAP 2310 as a software specific and project oriented supplement. 
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CHAPTER 1 INTRODUCTION 


1.1. PURPOSE 


This publication specifies the project oriented requirements to manage the quality of 
the software development process. Both managerial and technical processes must 
be addressed in order to: 


a. 


b. 


establish visibility of the software development process; 


detect software quality problems as early as possible in the software life 
cycle; 


provide quality control data for the timely implementation of effective 
corrective action; 


confirm that quality is engineered in during the software development 
process; 


provide assurance that the software produced conforms to contractual 
requirements; 


ensure that appropriate software support is provided to activities at the 
system engineering level, if required by the contract; and 


ensure that the safety and security conditions of the project are 
addressed. 


1.2. APPLICABILITY 


1. When referenced in a contract this AQAP shall apply to: 


a. 


b. 


all cases where software development is undertaken; 


all cases where non-deliverable software is developed or employed 
under the contract (to the extent specified in paragraph 2.2.4.8); 


all cases where software maintenance is part of the contract, in order to 
avoid uncontrolled, hidden development activities, which could have 
unforeseeable or detrimental consequences on the quality of the 
software product; 


all cases where off-the-shelf software is to be delivered (to the extent 
specified in paragraph 2.2.4.7); and 
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e. all cases relating to the development of the software element of 
firmware. 


If the contract addresses only "partial" software development or maintenance 


activities, then the related requirements of this publication shall also apply (e.g. 
software replication activities, software activities during system integration, software 
requirements definition, software archiving and storage services, Sub-supplier 


management activities etc.). 


3. 


This publication is intended for use with AQAP 2110 or AQAP 2310 as a 


software specific and project oriented supplement. Where there is any conflict 
between the requirements of AQAP 2110 (or AQAP 2310) and this publication for 


software, the requirements of this publication shall prevail. 


4. 


publication, the Contract requirements shall prevail. 


5: 


If any inconsistency exists between the Contract requirements and _ thi 


IS 


For competitive software acquisition this publication can also be used for the 


specification of requests for proposals and the evaluation of proposals. The 
provisions of this publication can also apply to Government Agencies performing 
software development or maintenance. 


1.3. 


1; 


1.4. 


REFERENCED DOCUMENTS 


AQAP 2110 Edition 3 "NATO Quality Assurance Requirements for Design, 
Development and Production". 


AQAP 2310 Edition A Version 1 "NATO Quality Management System 
Requirements for Aviation, Space and Defence Suppliers". 


ISO 9000: 2005 “Quality management systems — Fundamentals and 
Vocabulary". 


ISO/IEC 25010: 2011 “Systems and software engineering -- Systems and 


software Quality Requirements and Evaluation (SQuaRE) -- System and 
software quality models”. 


DEFINITIONS AND ACRONYMS 


1.4.1. Definitions 


The applicable definitions of ISO 9000 or AQAP 2110 (or AQAP 2310) apply to 
terminology used in this publication. Where definitions in ISO 9000 or AQAP 2110 (or 
AQAP 2310) and this publication differ, the definitions in this publication shall apply. 
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1. Control 

The activity to detect differences between an actual and planned result/process, and 
to cause changes in a process or a product which reduce the detected differences to 
a defined level. 


a Evaluation 

A systematic determination of the extent to which an entity meets its specified 
criteria. 

Notes: 

a. The term "entity" includes product, activity, process, organization or person; 

b. Evaluation of the activity or process may occur in parallel with development, or 
may be deduced as the result of verification of the software product; 

c. Evaluation of the activity or process can be performed by monitoring, auditing, 
process qualification or by establishing and documenting whether or not they 
conform to specified criteria. 


3. Firmware 
The combination of a hardware device and computer instructions or computer data 
that reside as read-only software on the hardware device. 


4. Method 
A set of rules for solving a problem. 


a Non-deliverable Software 
Software that is not required to be delivered under the contract but may be used in 
the development of software. 


6. Off-the-shelf Software 

Deliverable software that is already developed and usable as is, or with modification. 
Off-the-shelf software may be referred to as reusable software, Government 
furnished software, or commercially available software depending on its source. 


7. Process 

The interaction of personnel, equipment, material and procedures aimed at providing 
a specified service or producing a specified product. 

Each process is a defined set of one or more activities or tasks which can be 
accomplished in a finite period of time. Each process can be broken down into 
activities which are characterized by quantifiable inputs and outputs which can be 
measured, controlled and improved. 


8. Software Development Model 
A simplified, abstract representation of the software development process (process 
behaviour and results) used for planning and control purposes. 


9. Software Development Process 


The process by which user needs/requirements are translated into a software 
product. 
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10. Software Life Cycle 

A framework containing the processes, activities and tasks involved in the 
development, operation and maintenance of a software product, spanning the life of 
the system from the definition of its requirements to the termination of its use. 


11. Software Quality Characteristics 

A set of attributes of a software product by which its quality is described, verified and 
validated. A software quality characteristic may be refined into multiple levels of sub- 
characteristics. 

Note: According to the International Standard ISO/IEC 25010: 2011, software quality 
may be evaluated using the following eight characteristics: Functional suitability, 
Performance efficiency, Compatibility, Usability, Reliability, Security, Maintainability, 
and Portability. 


12. | Software/Software Product 
Computer programs, procedures, rules, associated documentation and data 
pertaining to the operation of a computer system. 


13. Software Tool 
A computer program used to help develop, analyze, evaluate, verify, validate or 
maintain another computer program or its documentation. 


14. Validation 

Confirmation by examination and provision of objective evidence that the particular 
requirements for a specific intended use are fulfilled. 

Notes: 

a. Validation is normally performed on the final product under defined operating 
conditions; 

b. Multiple validations may be carried out if there are different intended uses. 


15. Verification 

The process of determining and obtaining objective evidence whether or not the 
products of a given phase of the software development process fulfil the 
requirements established during the previous phases. 

Notes: 

a. Verification can be performed by reviewing, inspecting, testing, checking, auditing 
or otherwise establishing and documenting whether or not products conform to 
specified requirements; 

b. A phase in this context does not imply a period of time in the development of a 
software product. 


1.4.2. Acronyms 
The following acronyms appear in this document: 


Cl Configuration Item 
SCl Software Configuration Item 
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EVV Evaluation, Verification and Validation 
SCM Software Configuration Management 
SPQP Software Project Quality Plan 

sSQs Software Quality System 
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CHAPTER2 REQUIREMENTS 


2.1. SOFTWARE QUALITY SYSTEM (SQS) 


1. The Supplier shall apply a documented, effective and efficient SQS to the 
project. The SQS can be an integrated part of a general quality system, but shall be 
comprised of a comprehensive, integrated quality management process. This 
process shall be applied throughout the contract, ensuring that quality is designed in 
as the software development progresses. 


2. By correlation of budget and schedule deviations with quality information, the 
SQS shall also provide for the timely detection and correction of any negative 
influence on quality, thus minimizing technical risk. 


3: Provision shall be made for the periodic and systematic review of the SQS by, 
or on behalf of, Supplier's top management to ensure its effectiveness. 


2.2. PROJECT SOFTWARE QUALITY MANAGEMENT ACTIVITIES 
2.2.1. General 


12 To achieve visibility and control of the software development project the 
Supplier shall plan and implement effective software quality management activities. 


2: The Supplier shall undertake a formal contract review to ensure all the 
contractual requirements are defined and to determine the necessary management 
and technical processes which need to be planned and implemented. 


3: Based on contract requirements, the rules and procedures of the SQS and the 
specific project requirements, the software quality management activities shall: 


a. establish/identify, refine and allocate requirements to software products 
and configuration items (CIs). See para 2.2.3. 


b. establish and implement managerial and technical processes to 
develop, and build quality into the software. See paras 2.2.4/2.2.5. 


c. establish and implement procedures to verify and validate the quality of 
the software products and to evaluate processes and activities, 
including non-deliverable software, that impact the quality of the 
software products. See para 2.2.6. 


d. establish and implement procedures for risk management. The Supplier 
shall identify, analyze, prioritize and monitor the areas of the project 
that involve potential technical, cost or programme risk. The aim of risk 
management shall be to eliminate or minimise risk. 
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4. The software quality management activities shall call upon existing standards 
and procedures in the organization's SQS. When this is not the case a justification 
shall be provided to the Acquirer. 


5. The software quality management activities shall be documented in the 
Software Project Quality Plan (SPQP). See para 2.2.2. 


6. Provision shall also be made for the evaluation of the software quality 
management activities by the Acquirer, who may disapprove them. 


2.2.2. Software Project Quality Plan (SPQP) 


1. The Supplier shall document the software quality management activities as 
related to the Project in a SPQP. The SPQP may be a discrete document, or part of 
another plan that is prepared under the contract. The SPQP shall carry the signature 
of approval of those organisational elements having responsibilities identified in the 
SPQP, and be placed under configuration control. 


2. If stipulated in the Contract, the SPQP shall be offered to the Acquirer for 
agreement. Once agreed by the Acquirer the SPQP shall form part of the Contract. 
Any subsequent amendment to the agreed plan shall be subjected to the defined 
change control procedures agreed with the Acquirer and detailed in the SPQP. 


3. The SPQP shall address all the requirements of, and include or reference all 
procedures necessary for the fulfilment of the requirements of this Standard. If not 
specifically requested the information may be presented in the Plan in any sequence 
and format. 


4. The SPQP shall be used by the Supplier as a current baseline to define the 
activities to monitor and control the quality of the software project. The SPQP shall 
be reviewed and updated at pre-defined milestones during the project as new 
definitions and development details become known. 


2.2.3. Identification and Review of Software Requirements 


1. The Supplier shall identify the software requirements and development 
constraints. 


2 If a software requirement review has not been performed as part of system 
development, it shall be an initial step in the software development process and be 
prescribed in the SPQP. 


3: The review shall verify that software requirements are complete, consistent, 
unambiguous, traceable, feasible and can be validated. 


4. After the completion of the software requirements review, the software 


requirements specifications shall be formally approved by responsible authorities and 
shall be subject to configuration management. 
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5. If software requirement specifications are developed by the Supplier as part of 
a system contract, the software requirements shall be offered to the Acquirer, who 
may disapprove them, subject to the conditions of the contract. 


6. The software requirements specifications shall include a clear and precise 
definition of the design constraints and of the essential software quality 
characteristics. 


As The SPQP shall identify what standards or guides apply to the format and 
content of the software requirements specifications. 


8. Any uncertainty with the interpretation of the contractual software 
requirements shall be brought to the immediate attention of the Acquirer. 


2.2.4. Management 

2.2.4.1. Software Development Process 

1. The Supplier shall apply a development model which breaks down the 
development process into partial processes, and which satisfies the following quality 


related criteria: 


a. reduces the complexity of the development process to ensure visibility 
and control; 


b. describes software and system integration; 

c. describes the software system architecture; 

d. makes use of recognized software engineering practices; 

e. utilizes data feedback from previous designs; 

describes the activities and their expected results clearly; 

g. identifies tasks which are critical to quality and project success; 


h. defines and chronologically assigns control points at which the correct 
course of the process and the correct transfer of results can be verified; 


i. describes how unplanned activities will be controlled; 
provides unambiguous start and end criteria for all processes; 


k. provides clear identification and allocation of all quality functions within 
the project specific organizational structures; 


uses proven and qualified constructive and analytical quality measures; 


m. provides quality data for the effective management of the development 
process; 
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n. relates planning, monitoring and release activities to software 
engineering activities; and 


o. reduces the risk by using computer resources to free people involved in 
the software development process from error prone, repetitive 
activities. 


2: Any changes to development models, adopted during the project, need to be 
recorded in the project plan. 


2.2.4.2. Organization 


‘le The Supplier shall define and implement the organizational structure, 
responsibilities, authorities and the inter-relationship of organizational elements and 
groups that plan, direct, perform and control activities affecting software quality. 


2) Personnel performing software quality evaluations, verifications and 
validations shall have the resources, responsibility, authority, and technical expertise. 
They shall also have an appropriate level of independence from the person(s) who 
developed the software product or performed the activity being 
evaluated/verified/validated, to permit objectivity and to cause the initiation of 
corrective action. 


S. A representative shall be appointed with the necessary authority to ensure all 
the requirements of this publication are met. 


2.2.4.3. Non-conforming Software 
The Supplier shall: 


a. establish and maintain control of any software that does not conform to 
specified requirements, to ensure that unintended use or delivery is 
prevented; 


b. notify the Acquirer of any non-conforming products received from Sub- 
suppliers that have been subject to Government Quality Assurance 
(see para 2.2.4.5); 


c. provide controls, agreed by the Acquirer, for the identification and 
segregation of non-conforming software; 


d. comprehensively document the nature of the non-conformances and 
the functions affected; 


e. document the procedures for the disposition of non-conforming 
products; and 


f. notify the Acquirer of any intention to deliver non-conforming software. 
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2.2.4.4. 


Corrective Action 


i: The Supplier shall define and implement a corrective action process to ensure 


that: 


all problems detected in processes and products are documented, 
assessed for their validity, and analyzed to identify trends; 


problems are reported to a level of management which has the 
necessary authority to ensure timely corrective action is taken; 


prompt and effective action is taken to resolve problems and correct 
adverse trends, and status is tracked and reported; 


. feedback is provided to the Acquirer as required by the contract or the 


SPQP; 


data for measuring and predicting the quality of the software 
development process is provided; and 


records are maintained and made available to the Acquirer for the life 
of the contract or as specified within the contract. 


Z. The corrective action process shall address both technical problems and 
managerial problems encountered, with the aim of preventing recurrence. 


2.2.4.5. 


1. For 


Sub-supplier Management 


sub-contracted software specifically developed for the contract 


(deliverable or non-deliverable) the main Supplier shall: 


a. 


b. 


apply effective Sub-supplier selection procedures; 


define the software product/service and quality management 
requirements, including the requirements for a Sub-supplier's SPQP; 


conduct verifications/validations/evaluations of sub-contracted items / 
processes, including the Sub-supplier's SPQP; 


define how changes are to be processed, including the Sub-supplier's 
participation; and 


define the actions available to the Supplier should the Sub-supplier not 
be in conformance with the contract or SPQP. 


2. Provision shall be made for Government Quality Assurance at the Sub- 
suppliers facilities when requested by the Acquirer. When the Acquirer determines 
that Acquirer verification/validation/evaluation of the Sub-suppliers items/processes is 
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necessary, the Supplier shall provide for this in the purchasing document. Copies of 
the purchasing document together with the relevant technical data shall be provided 
to the Acquirer on request. 


2.2.4.6. Software Configuration Management (SCM) 


1. The Supplier shall define and implement a SCM process to maintain 
integrity and traceability of the software product(s) during development. The SCM 
activities and procedures shall ensure that uncontrolled changes are prevented, and 
shall provide planned and released baselines as a reference and prerequisite for 
verification, tracing and controlling software quality. 

Specifically, the Supplier shall define and implement: 


a. procedures to identify, name and record the physical, functional and 
quality characteristics of intermediate and final items to be controlled 
(e.g. documentation, executable code, source code, program listings, 
data bases, specifications, test cases, plans) and their structures at 
each project control point. Elements of the development and support 
environment (compilers, development tools, operating systems, test 
beds) shall also be part of the Software Configuration Item (SCI) 
structure; 


b. procedures to request, evaluate, approve/disapprove and implement 
changes (error correction and enhancement) to baselined SCls; (The 
practice of software patching shall be restricted to very exceptional and 
temporary situations. It shall not be done, without the knowledge and 
agreement of the Acquirer. Configuration control of patches shall be 
prescribed in a specific procedure.) 


c. procedures to record and report the status of project SCls; 


d. audits and reviews for the determination to what extent the SCls reflect 
the required physical, functional, and quality characteristics (see also 
2.2.6), and for establishing a baseline; 


e. procedures to control interfaces of project SCls with items outside the 
direct scope of software development (system, hardware, human, 
support software); and 


f. procedures to coordinate changes to externally developed software 
items (see also 2.2.4.5) and to incorporate those changes into the 
project. 


2: Changes to the software requirement specifications shall be evaluated for 
cost, technical and schedule impact, and be communicated to all affected parties. 
Changes that will affect functional performance shall only be implemented with 
acquirer approval. 
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3: The Supplier shall also identify the software tools, techniques and 
equipment which are necessary to implement SCM activities (see also 2.2.5), and 
allocate responsibilities and authorities for SCM activities to organizations and 
individuals within the project structure. 


2.2.4.7. Off-the-shelf Software 


1. If the Supplier employs deliverable off-the-shelf software, he shall ensure 
that: 


a. its usability is unaffected by any existing data protection rights; 


b. objective evidence exists, prior to its use, that the software will perform 
the required functions; 


c. the software is placed under configuration management; and 


d. the software is documented in accordance with the requirements of the 
contract and this publication. 


Des If deliverable off-the-shelf software is modified during the development 
process, such software shall then be treated as software under development and 
shall be subject to the requirements of this publication. 


3. If the Supplier establishes that off-the-shelf software supplied by the 
Acquirer is not acceptable for use, he shall promptly report the reasons for its 
unacceptability to the Acquirer and negotiate with him the remedial actions to be 
taken. 


4. The Supplier shall advise the Acquirer when off-the-shelf software is to be 
incorporated into the software product. 


2.2.4.8. Non-deliverable Software 


If the Supplier employs non-deliverable software in the development of the 
deliverable software, then he shall ensure that: 


a. objective evidence exists, prior to its use, that the software will perform 
the required functions; and 


b. the software is placed under configuration management. 


2.2.4.9. Quality Records 


All records that demonstrate the achievement of quality shall be made available to 
the Acquirer. 
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Quality records shall: 


a. provide objective evidence that the software development process was 
performed in conformance with Acquirer requirements and recognized 
software engineering practice as detailed in the SPQP; 


b. provide historical or reference data that may be used to detect long 
term trends and quality deficiencies in the development process; and 


c. be traceable to their controlling procedures. 


2.2.4.10. Documentation 


‘A The Supplier shall identify the software documentation, including Quality 
Records to be retained together with a recommendation for the retention period. The 


Supplier shall sta 


te the methods and facilities to be used to assemble, safeguard and 


maintain this documentation. 


Zt Applicable software licenses shall cover the intended use of the software 


product. 


2.2.4.11. Handling and Storage of Software Media 


The Supplier shal 


a. soft 


| ensure that: 


tware is stored so that retrieval is assured; 


b. a system is in place that allows access to software only through an 


autl 


horization process and which makes software accessible only to 


those with a demonstrable need to know of, or use such software; 


c. the 
soft 


environment is controlled so that the physical media on which the 
tware is stored do not degrade; 


d. secondary secure storage and retrieval are provided for critical 


soft 


tware and copies of baselined software. 


2.2.4.12. Replication and Delivery 


The Supplier shal 


a. the 
soft 


b. the 


| ensure that: 


replication process to generate multiple customized versions of 
tware is under control; 


process of software release including the method of issuing multiple 


customized versions of software, is documented, reproducible and 
under control; 
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c. procedures are implemented for marking, handling, storing, preserving 
and packing software, such that its integrity is assured until it is 
delivered to the destination specified in the contract; 


d. procedures are implemented for the certification of the conformity of the 
software to the contract requirements; 


e. procedures are implemented for the keeping of records relating to the 
distribution of deliverable items. 


2.2.5. Software Engineering 


Is For the software development and/or maintenance activities the Supplier 
shall employ recognized software engineering methods, tools, resources and 
procedures. The Supplier shall also identify and standardize specific conventions for 
any graphical or formal linguistic notations. The methods, tools, standards and 
procedures used shall support the software lifecycle to: 


a. express software requirements including quality characteristics; 

b. translate the Acquirer/user oriented software quality requirements into 
software engineering oriented characteristics and allocate these to the 
appropriate level of design; 

c. ensure traceability at all design and implementation levels; 


d. minimize errors; and 


e. support evaluation/verification/validation during software development 
and/or maintenance. 


2: The methods and procedures used shall be evaluated and documented, 
and shall support the recognized principles and concepts of software engineering 
that influence software quality. Software tools shall be validated to confirm their 
performance and integrity by a defined method. 

2.2.6. Evaluation, Verification and Validation (EVV) 


‘Ie The Supplier shall plan, define and implement: 


a. a process for evaluation of software methods, techniques, procedures, 
tools and activities; 


b. a process for verification and validation of software items and software 
products; 


c. a process for the provision of follow-up action to ensure that necessary 
changes are made; and 
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3: 


4. 


goals. 


2.2.6.1. 


ls 


d. 


a process to determine the required level of reverification in the case of 
error correction or change to the requirement/design. 


The EVV process shall define: 


e. 


EVV activities and their sequence in relation to phases, milestones and 
time schedule; 


the organizational roles, responsibilities and authorities for the 
execution of EVV activities (see also 2.2.4.2); 


EVV objects (e.g. requirements/development documents, software 
products, development processes, methods, procedures, source code, 
object code); 


the criteria to perform EVV; 
specific EVV methods, standards, techniques, tools and facilities; 
the type of EVV methods to be used e.g. test, review, audit; and 


the EVV documentation to be produced (specific plans and procedures, 
EVV records and reports). 


As an integral part of the EVV process the Supplier shall develop/select 
and implement quantitative and/or qualitative measures to evaluate/verify/validate the 
software quality characteristics specified in requirements specifications. 


Quantitative/qualitative measures (metrics) shall also be applied to 
manage and control the software development process for the software product 
under contract. Such measures shall enable identification of the current level of 
performance, the taking of remedial action and the establishment of improvement 


Testing 


As an integral part of the EVV process the Supplier shall plan, define and 
implement a test programme. Consideration shall be given to: 


a. software item, integration, system and acceptance testing; 


b. test environment, tools and test software; 


c. user documentation; and 


d. personnel required and associated training. 
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2: The Supplier shall undertake a review of test requirements and criteria for 
adequacy, feasibility, traceability and ambiguity. Test specifications shall be prepared 
which define test cases, required test data and expected results. 


3: The Supplier shall define and implement measures to control test activities 
which include: 


a. the establishment, documentation and verification, as necessary, of the 
configuration of the software to be tested, together with any associated 
hardware; 


b. the maintenance of test related documentation to allow test 
repeatability; 


c. confirmation that tests are conducted in accordance with approved 
plans, specifications and procedures; 


d. provision for certification that test results are actual and valid; and 


e. provision for review and certification of test reports. 


4. The Supplier shall report unusual difficulties found during test to the 
Acquirer. 

2.2.6.2. Reviews 

“les The Supplier shall define and implement review procedures to verify that 


contractual software requirements are being met. 

2: Reviews shall be identified in, and form an integral part of the overall 
software development process. Reviews shall be planned, conducted systematically 
and be critical of the item under review. 

3: Review procedures shall include provisions for: 


a. describing the objectives of each review; 


b. identifying the functions, authorities and responsibilities of personnel 
involved in the reviews; 


c. recording review findings; and 
d. ensuring that actions resulting from reviews are monitored to ensure 


timely completion. 


4. All software documentation generated under the contract shall be reviewed 
and approved for adequacy by authorized personnel prior to issue. 
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2.2.7. Maintenance 
rT: When, after initial delivery and installation, software maintenance is a 
specified requirement, the Supplier shall define and implement procedures for 


performing this activity. The procedures shall include provision for verifying and 
reporting that the maintenance carried out meets specified requirements. 


Pa Consideration shall be given to: 
a. the work to be done; 
b. the procedures to be employed; 
c. the records and reports to be produced; 
d. the responsibilities of the Supplier and his interface with the Acquirer; 


e. the configuration management activities, including the identification of 
he initial status of the product to be maintained; 


f. the methods for dealing with the reporting, analysis and resolution of 
problems; and 


g. testing and acceptance of modifications. 


2.3. HUMAN RESOURCES 


‘Is Personnel performing specific assigned tasks (Outsourced labour or 
company employees) shall be qualified on the basis of appropriate education, 
training and/or experience as required. 


2: Appropriate records shall be maintained. (See para 2.2.4.10). 


2.4. ACQUIRER ACCESS AND INVOLVEMENT 


“les The Supplier shall provide the Acquirer with the accommodation and 
facilities required for the proper accomplishment of his work and with all necessary 
assistance for the evaluation of the software quality program and the verification and 
validation of products. 


a The Acquirer shall have right of access to any of the Supplier's or Sub- 
supplier's facilities where any part of the contracted work is being performed. The 
Acquirer shall be afforded unrestricted opportunity to verify conformance of the 
supplies with contract requirements. The support tools necessary for evaluation, 
verification and validation purposes shall be made available for reasonable use by 
the Acquirer. 
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3: The Supplier shall be aware that Acquirer evaluation, verification and 
validation shall not constitute acceptance, nor shall it in any way replace EVV 
activities by the Supplier or otherwise relieve the Supplier of his contractual 
responsibilities. 
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ANNEX A 


INDEX 


The index below is aimed to help, when searching for a specific subject in AQAP 
2210. Only a limited number of words are chosen and this should not be interpreted 
as a list of priority. The words are referenced to the paragraph in which they appear. 
They may appear more than once. The "main requirement paragraph" is underlined. 


Paragraph 1.4 is Definitions and Acronyms. 
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Software development process 
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ANNEX A — Guidance for a Software Project Quality Plan (SPQP) 


FOREWORD 


This document has been prepared and issued to provide information and guidance on the 
application of: AQAP-2210 "NATO Supplementary Software Quality Assurance 
Requirements to AQAP-2110 or AQAP-2310". 


It aims to contribute to commonality of interpretation of these requirements between Supplier 
and Acquirer. It is not intended as a procurement document. Its content has no legal or 
contractual status, nor does it supersede, add to or cancel any of the AQAP-2210 
requirements. Copies of this document may be made available to industry to facilitate the use 
and understanding of AQAP-2210. 


Each paragraph (and subparagraph) of AQAP-2210 is listed in this document only with his 
title in bold italic, followed by the related guidance (or by the sentence "No guidance 
required"). The guidance offers some suggestions as to subjects and factors to be 
considered. 


Because of the multiplicity of conditions that can exist (dependent on such factors as the type 
of work or process, the devices used and the skill of personnel involved), this guidance should 
not be considered as all-encompassing, nor should it be considered as imposing specific 
means or methods for meeting contract requirements. Managers must be aware that other 
means or methods could be used to meet these requirements. 


The fundamental requirements of AQAP-2210 are mandatory for all software projects, but 
the sub-level application of tools, methods and procedures can be implicitly tailored to the 
needs of individual projects. 


This publication supersedes the Annex E to the AQAP-2009 Edition 3. 
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CHAPTER 1 INTRODUCTION 


1.1. PURPOSE 


In addition to the requirements (a. through e.) strictly related to the software development 
process, this Publication also addresses the system-software relationship. The additional 
requirements (f. and g.) provide for the meaningful participation of software engineering in 
system engineering, and for addressing the system/software critical issues, like safety and 
security. 


1.2 | APPLICABILITY 


No guidance required. 


1.3. REFERENCED DOCUMENTS 


No guidance required. 


1.4 DEFINITIONS AND ACRONYMS 


No guidance required. 
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CHAPTER 2 REQUIREMENTS 


2.1 SOFTWARE QUALITY SYSTEM (SQS) 


AQAP-2210 normally presumes the existence of a documented Software Quality System 
(SQS); the SQS includes not only the technical processes of software development but also 
the managerial processes. 


The company-wide SQS should address the range of software that the Supplier produces. 
Different methods, procedures and tools may be called for dependent on the type of 
application, size of project, number of people involved etc. 


Review of the SQS is defined as a periodic, systematic and documented evaluation of the 
status and adequacy of the system elements. Such a review is conducted by or on behalf of 
top management to ensure that their objectives are reached, and to reveal non-conformances 
or irregularities in the system elements that require improvement. 


For the SQS to be effective it should support the requirements of AQAP-2210 and any 
additional requirements imposed by the contract. 


2.2 | PROJECT SOFTWARE QUALITY MANAGEMENT ACTIVITIES 


2.2.1 General 


The Project Software Quality Management Activities should comprise the planning and 
implementation activities necessary for the successful execution of the project. The project 
activities mentioned in paragraphs 2.2.1 a, b, c and d are elaborated in paragraphs 2.2.3 
through 2.2.7. Guidance on these activities is given on each subparagraph. 


The depth of Project Software Quality Management Activities will be influenced by the 
contractual requirements and constraints, like complexity, criticality, size, Acquirer 
involvement etc. Therefore, as a prerequisite to the planning of the activities, the Supplier 
should undertake a formal contract review, to ensure all requirements and constraints are 
clearly defined and understood. 


Evaluation of the activities by the Acquirer, should initially make use of objective evidence of 
the Supplier's own reviews. Where no objective evidence exists that such reviews have been 
conducted, it should be regarded as a serious quality system shortcoming and consequential 
risk. 
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2.2.2 Software Project Quality Plan (SPQP) 


The SPQP and its contents should be recognized by Acquirer and Supplier as an indication 
of the understanding, commitment and compliance with the quality requirements of the 
contract. 


Suppliers should begin to plan their quality related activities at the earliest possible phase of 
the contract. 


The SPQP should address "contract-specific" quality activities and should not be a reiteration 
of the SQS requirements, as detailed in the Supplier's Quality Manual/Documentation. 
However, reference to these requirements in the SPQP may be necessary. 


An SPQP may be required in response to an Invitation-to-Tender / Request-for Proposal, or 
under the contract, and should be prepared as a precursor to the software development 
process. 


A possible layout for the SPQP may be found at Annex A. 


2.2.3 Identification and Review of Software Requirements 


The software requirements may be derived from an expressed need (but not necessarily 
specified) by the Acquirer. Often the Supplier does not fully understand the Acquirer's 
problem and field of application; both contractual parties may work together to come to a 
formal contractual agreement on what the completed software must do. 


The key to achieving effective software development is for both the Supplier and Acquirer to 
achieve a common understanding of the requirements. Therefore, the Supplier should ensure 
that the software requirements are described in such a way that their interpretation is not in 
doubt. Any omissions, misunderstandings or inconsistencies in the requirements should be 
addressed as early as possible in the software development process when they are easier to 
correct. The Supplier should also be satisfied that each requirement is defined in such a 
manner, that its achievement can be ultimately subjected to validation by a prescribed 
method. If this is in doubt, the matter should be brought to the attention of the Acquirer. 


Often the software requirements are derived from higher level (i.e. system or subsystem 
requirements), in that case the task at hand consists in ensuring that all applicable higher 
level requirements have been correctly translated into software requirements and no new 
requirements have been introduced. 


"Development constraints" are restrictions on the development process, which shift greater 
design responsibility to the Organization setting the restrictions and need to be separated 
from the software product characteristics. Examples are: Design standards and conventions, 
languages, computer hardware and Acquirer supplied software. 
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Consideration should be given to the provision of training of personnel (both Acquirer and 
Supplier). 


Definitions of software quality characteristics can be found in ISO/IEC 25010: 2011 
(confirmed in 2017). These include Functional suitability, Performance efficiency, 
Compatibility, Usability, Reliability, Security, Maintainability and Portability. 


2.2.4 Management 


2.2.4.1 Software Development Process 


The software development has a strong impact on the quality of the software product. 
Software development models are simplified, abstract representations of a systematic 
approach to the software development process and, together with methods and tools, are the 
most important quality management elements. 


Development models are a basis for detailed planning of project software quality 
management activities, including time and budget aspects, and support continuing 
improvement of the software development process. 


The models structure the processes in logical and coordinated activities and tasks, and 
clearly relate development activities to the associated evaluation activities. 


There are several types of development model, e.g. Waterfall Model, Spiral Model etc.; 
AQAP-2210 gives the Supplier freedom in the choice of development model. The selection, 
definition and application of a specific model depends on the complexity, criticality and type 
of software to be developed. Whatever model is selected, it may be tailored to meet the 
specific contract requirements. However, the model should achieve the issues mentioned in 
paragraph 2.2.4.1 (a. through 0.) of AQAP-2210. Where possible, account should be taken 
of International or National standards defining these models. 


It should be noted that whilst paragraph 2.2.4.1 is under the parent paragraph 2.2.4 
(Management), the software development process also includes the technical processes 
described in paragraph 2.2.5 (Software Engineering) and paragraph 2.2.6 (Evaluation, 
Verification and Validation). 


The model should clearly describe all the primary processes, e.g. design, coding, testing etc., 
together with all the supporting and organizational processes, e.g. project management, 
quality management, configuration management etc., undertaken throughout the software 
life-cycle. The description of the processes should not only include the identification of the 
tasks, but also the roles (architect, tester, etc.), inputs, outputs, the start and end criteria, 
control points, related measurement (when applicable) and all the technical and managerial 
aspects. This is in order to reduce the complexity of the software development process, thus 
giving improved visibility, integrity and control of the software product itself. 
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A developed software integration strategy should include verification criteria for software 
items, consistent with the software design and the prioritized software requirements: 
i. items are developed that ensure compliance with the software requirements 
allocated to the items; 
ii software items are verified using the defined criteria; 
iii software items defined by the integration strategy are produced; 


iv results of integration testing are recorded; 

v consistency and traceability are established between software design and 
software items; 

vi a regression strategy is developed and applied for re-verifying software items 


when a change in software units (including associated requirements, design 
and code) occur. 


2.2.4.2 Organization 


It is important to define the inter-relationship of organizational elements and groups, since 
activities of the development process may overlap and be executed iteratively. It is also 
important that the organizational structure indicates the co-operation and consultation 
between elements or groups, and also indicates the point(s) of contact with the Acquirer. 


The degree of independence required for personnel performing evaluations, verifications and 
validations, may depend upon the circumstances of the particular contract and/or Supplier 
concerned. In most instances suitably, independent personnel may be found amongst the 
peers of those who developed the software product or performed the activity being subjected 
to evaluation, verification or validation. Sometimes it may be necessary to seek such 
personnel within other areas or organizations, internal or external to that of the Supplier. 
Where special independence requirements pertain, such as for safety critical software, these 
should be defined in the contract. 


A determination of the necessary independence of the verification effort should be required 
based on the potential of an undetected error in a system or software for causing: 

i death or personal injury; 

ii mission failure; 

iii catastrophic equipment loss or damage; 


iv the maturity of and risks associated with the software technology to be used; 
v financial loss. 
2.2.4.3 Non-conforming Software 


Non-conforming software should be clearly identified as such and segregated from 
conforming software. Once "conforming" software is released and made available for use, 
e.g. to test areas or placed in the software library, its status should be clearly indicated and 
made known. Upon becoming non-conforming software, e.g. after failing a test or a confirmed 
customer fault report, it should be segregated by clearly indicating its non-conforming status 
and taking appropriate action to control access to the software. 


Edition A Version 1 


AQAP-2210-SRD.1 


2.2.4.4 Corrective Action 


The primary aim of the corrective action process should be to prevent the recurrence of a 
problem. It will also be a source of data for the review of the SQS. The analysis of problems 
should consider the effectiveness of any processes involved, be they technical or managerial. 


2.2.4.5 Sub-supplier Management 


The main Supplier is responsible for ensuring that sub-contracted products and services 
comply with the requirements and conditions of the main contract, even if the entire software 
package is sub-contracted. 


The Supplier should select Sub-suppliers, using an appropriate procedure, based on their 
ability to meet sub-contract requirements, including quality. The Sub-suppliers previously 
demonstrated performance should also be taken into account. 


The Sub-supplier's SPQP should be related to the main Supplier's SPQP. This relationship 
of plans is necessary for configuration management and specifically to coordinate changes 
to configuration items. 


2.2.4.6 Software Configuration Management (SCM) 


In software development and/or maintenance, a strong relationship exists between SCM and 
software quality assurance. Without a disciplined SCM process, one of the means for quality 
assurance is missing. 


Configuration management is a discipline for identifying, controlling, tracking and auditing the 
versions of each software configuration item. SCM should be applied in a cost-effective 
manner, in terms of organization, methods, tools and procedures, whilst ensuring the 
necessary integrity and traceability of the software product. Configuration management can 
be automated or undertaken by manual methods. 


Temporary changes to delivered software, sometimes known as patches, should be strictly 
controlled. Where such changes are introduced into software they should be carried out in 
accordance with defined procedures. In any event, follow-up action should confirm the validity 
of the change and where appropriate formally introduce it under normal configuration 
management procedures. 


2.2.4.7 Off-the-shelf Software 


The reason why off-the-shelf software should be placed under Configuration Management is 
because it affects the integrity of the developed software. This is true, whether it is a 
component of the software under development or a tool to assist the development of such 
software. 
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Off-the-shelf software by definition includes "government furnished software" (see paragraph 
1.4.1, item 6, of AQAP-2210). Government furnished software places constraints on the 
Supplier in terms of development freedom and responsibility. 


The evaluation and validation of the ability of off-the-shelf software to perform the required 
functions may include such considerations as Intellectual Property Rights, licensing 
arrangement and configuration management controls. 


The Supplier should be able to provide objective evidence (e.g. validation reports, 
configuration reports, etc.) that the use of off-the-shelf software has been evaluated and is 
under control. 


Documentation requirements for off-the-shelf software may include functional and interface 
specifications. 


2.2.4.8 Non-deliverable Software 


Examples of non-deliverable software which may be employed in the development of 
deliverable software are: emulators, test harnesses and driver programs, stub routines etc. 


It is essential that all such software is placed under configuration management, since it 
directly affects the integrity of the developed software. 


2.2.4.9 Quality Records 


Quality records may be in the form of EVV reports, test results, corrective action reports etc. 
They can be the formal results of both main Supplier and Sub-supplier activities. 


2.2.4.10 Documentation 


There are several reasons for documentation retention, e.g. to: 
i facilitate the correction of faults; 
ii allow traceability of product; 
iii provide evidence in liability disputes; and 
iv allow for the re-creation of the software development environment. 


The Supplier should therefore identify all necessary documentation to allow the successful 
completion of such tasks. 


Documentation should include but not be limited to: 
i requirement specifications; 
ii architectural and design documents; 
iii user documentation; 
iv testing documentation; 
Vv quality records; and 
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vi software licenses, e.g. seats, number of platforms, number of users, reuse, 
interfaces, replication etc. 


Documentation can be in electronic or hard copy. 


2.2.4.11 Handling and Storage of Software Media 


Any media on which software is stored should be handled in such a way that the integrity and 
confidentiality of the stored information is assured. It is therefore necessary that activities 
likely to influence the quality are recognized and steps taken to avoid degradation of the 
material or the information. The Supplier should describe the storage, storage security, 
environment, access to and release from storage, in a procedure that also indicates how 
these activities are controlled. 


Software may be considered as "critical" because of its safety, security or other implications. 
However, it may also be considered as critical if, for example, its loss would seriously delay 
the successful completion of the software development programme. 


Adequate antivirus and firewall protection should be provisioned. 


2.2.4.12 Replication and Delivery 


No guidance required. 


2.2.5 Software Engineering 


Software engineering is the defined, documented and controlled, engineering discipline which 
develops the software products, with the use of methods, tools and procedures that are 
established and documented. The methods and procedures applied in software engineering 
should be consistent with the development model and criteria defined in paragraph 2.2.4.1. 


Software tools may be related to the specific methods or techniques identified or provide 
support to other aspects of the software life-cycle. Some tools may be phase-independent, 
for example those associated with configuration management or quality assurance activities. 


Software tool validation may entail one or more of the following: 
i certification provided by a recognized body that the tool has been subject to 
specified tests or validation processes; 
ii establishment, with the tool supplier, that the tool meets required criteria 
through the Supplier quality system and evidence of appropriate tests; 
iii identification of appropriate tests to be applied to the tool and any upgrades; 


iv monitored usage of the tool during support to the development of the software 
product; 
v feedback from a user group. 
-10- 
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To facilitate software product maintenance, the availability of longer-term support for software 
tools is an important aspect that should not be ignored in their evaluation. 


2.2.6 Evaluation, Verification and Validation (EVV) 


Although EVV is an integral part of the management and technical process (paragraphs 2.2.3, 
2.2.4 and 2.2.5), due to its importance in Quality Management, the EVV process is addressed 
in this discrete paragraph. 


Due to the inter-relationship of Evaluation, Verification and Validation, these activities should 
be planned as a whole. The allocation of resources and time, and the selection of methods 
and techniques should be done in such a way that the entire EVV process is optimized. 


The correct execution of EVV tasks has a considerable impact on the quality of the end 
product. This process requires, in general, the use of a considerable amount of resources, 
so that it should be carefully planned in terms of availability of qualified personnel, schedule, 
cost and test environment. 


The level of EVV should be tailored to the level of complexity and/or criticality of the software 
and to the requirements of the contract and should involve optimum use of existing 
techniques and standards available. 


This paragraph is also related, as a by-product, to the evaluation and improvement of the 
Software Quality System, e.g. it monitors the application of the established procedures and 
measures the correctness and efficiency of those procedures. This evaluation process is 
based on data provided by project groups and is a contract-independent activity (see 
paragraph 2.1). 


2.2.6.1 Testing 

In general, tests are much more effective the earlier they are addressed/conducted in the 
software development process. Planning for testing and the specification of tests should 
therefore take place as early as possible. 

During test planning, if required by the contract, consideration should be given to the 
involvement of Acquirer personnel in test activities. 


2.2.6.2 Reviews 


Software-related review activities may be known under various headings, including design 
reviews, peer reviews, walk-throughs, inspections, document reviews, desk checks etc. 


-11- 
Edition A Version 1 


AQAP-2210-SRD.1 


Experience has shown that significant software errors are introduced during the early phases 
of the software development process. Emphasis should therefore be placed on design 
reviews at these stages to promote the early detection and resolution of errors. 


2.2.7 Maintenance 


Software Maintenance is the process of maintaining software after initial delivery and 
installation, e.g. to correct defects, modify/adapt functions, or improve/augment performance. 


2.3. HUMAN RESOURCES 


Personnel performing specific assigned tasks shall be qualified on the basis of appropriate 
education, training and/or experience as required: 

i design methods; 

ii specific programming languages; 

iii tools, techniques; 

iv computer platforms and target environment. 


2.4 ACQUIRER ACCESS AND INVOLVEMENT 


No guidance required. 
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ANNEX A - Guidance for a Software Project Quality Plan (SPQP) 


In AQAP-2210 (paragraph 2.2.2) a Software Project Quality Plan (SPQP) is required and 
requirements for the SPQP may be found throughout AQAP-2210. 


As written in the same paragraph, the SPQP may be a discrete document or part of another 
plan that is prepared under the contract. 


A possible layout of the SPQP may be found below. It should be noted however, that itis a 
guide and that the SPQP, documenting the software management activities related to a 
specific project, is the sole responsibility of the Supplier. 


The requirements for evaluation of the software quality management activities by the 
Acquirer, are laid down in AQAP-2210, paragraph 2.2.1. 


If stipulated in the contract, the SPQP shall be offered to the Acquirer for agreement, as 
called for in AQAP-2210, paragraph 2.2.2. 


0 COVER SHEET 

The cover sheet should carry the signature of approval of those organizational elements 
having responsibilities identified in the SPQP. It may also indicate the name(s) of the 
organization(s) for whom the SPQP is prepared. 


1 INTRODUCTION 


1.1. Purpose 
A graphic presentation of the project may be included, or reference to where else it 
may be found. This could e.g. summarize the milestones and the numbers and 
positions of sites (or subsystems). 


1.2 Scope 

If the entire project is covered by plans, such as: 

- project management plan, 

- time schedule/milestone, 

- work breakdown structure, 

- list of deliveries, 

- risk assessment, 

- test plan and specifications, 

- installation handbook, 
reference to the plans can be made in paragraph 1.2 and the only reference needed 
in paragraph 1.4 may be AQAP-2210. 


1.3. Maintenance of the SPQP 
e.g. change control procedures. 


1.4 Referenced Documents 
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Relationship to other plans 
Definitions and Acronyms 


PROJECT DESCRIPTION 

Project Overview (or reference to where else it may be found) 
Assumptions 

Deliverable Products 


MANAGEMENT 

Software Development Process 
Organization 

Non-conforming Software 

Corrective Action 

Sub-supplier Management 

Software Configuration Management (SCM) 
Off-the-Shelf Software 

Non-deliverable Software 

Quality Records 

Documentation 

Handling and Storage of Software Media 
Replication and Delivery 


SOFTWARE ENGINEERING 
Software Engineering Environment 
Methods, Procedures, Standards 
Development Documentation 


EVALUATION, VERIFICATION AND VALIDATION (EVV) 
Testing 
Reviews 


MAINTENANCE 
HUMAN RESOURCES 


ACQUIRER ACCESS AND INVOLVEMENT 
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